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Abstract 

We construct a symplectic twist map / of the annulus that has an essential 
invariant curve T such that: 

• r is not differentiable; 

• the dynamic of /ip is conjugated to the one of a Denjoy counter-example. 
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1 Introduction 



The exact symplectic twist maps of the two-dimensional annuluqU were studied for a 
long time because they represent (via a symplectic change of coordinates) the dynamic 
of the generic symplectic diffeomorphisms of surfaces near their elliptic periodic points 
(see [5]). One motivating example of such a map was introduced by Poincare for the 
study of the restricted 3-Body problem. 

The study of such maps was initiated by G.D. Birkhoff in the 20's (see [3]). Among 
other beautiful results, he proved the following one (see [9] too) : 

Theorem. (G.D. Birkhoff) Let f be a symplectic twist map of the two-dimensional 
annulus. Then any essential curve that is invariant by f is the graph of a Lipschitz 
map. 

In this statement, an essential curve is a simple loop that is not homotopic to a 
point. 

Later, in the 50's, the K.A.M. theorems provide the existence of some invariant 
curves for sufficiently regular symplectic diffeomorphisms of surfaces near their elliptic 
fixed points (see [10], [2], [11] and [12j). These theorems provide also some essential 
invariant curves for the symplectic twist maps that are close to the completely inte- 
grable ones. These K.A.M. curves are all very regular (at least C^, see [9]). 

But general invariant curves for general symplectic twist maps have no reason to be 
so regular. The example of the simple pendulum (see [6]) shows us that an invariant 
curve can be non-differentiable at one point: the separatrix of the simple pendulum 
has an angle at the hyperbolic fixed point. In [9] and [1], some other examples are 
given of symplectic twist maps that have an non-differentiable essential invariant curve 
that contains some periodic points. 

In all these examples, the non-differentiability appears at the periodic points. A 
natural question is then: 

Question. Does a symplectic twist map exist that has an essential invariant curve 

that is non-differentiable at a non-periodic point? 

A related question is the following one, due to J. Mather in [7]: 

Question. (J. Mather) Does there exist an example of a symplectic C twist map 
with an essential invariant curve that is not and that contains no periodic point 
(separate question for each r G [1, oo] U {u)})? 



all these notions will be precisely defined is subsection 12.11 
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Let us point out that such an invariant essential curve cannot be too irregular : 

• firstly, BirkhofF theorem implies that this curve has to be the graph of a Lipschitz 
map; hence, by Rademacher theorem, it has to be differentiable above a set that 
has full Lebesgue measure; 

• secondly, I proved in pLj that this curve has too be above a Gs subset of T 
that has full Lebesque measur^. 

We will prove: 

Theorem 1. There exists a symplectic twist map f of the annulus that has an 
essential invariant curve T such that: 

• r contains no periodic points; 

• the restriction /|p is C^-conjugated to a Denjoy counter-example; 

• if J : T — 7- R is the map whose T is the graph, then 7 is at every point 
except along the projection of one orbit, along which 7 has distinct right and left 
derivatives. 

This lets open Mather's question for r > 2 and also the following question: 
Question. Does a symplectic twist map exist that has an essential invariant curve 
that is non- differentiable and that is such that the dynamic restricted to this curve is 
minimal? 

Before giving the guideline of the proof, let us comment on some related results. 

Finding invariant curves with no periodic points that are less regular that the con- 
sidered symplectic twist map has been a challenging problem for a long time. 
First at all, as an application of K.A.M. theorems, for a fixed diophantine rotation 
number and a 1-parameter smooth family (for example the standard family) such that 
the invariant curve disappears, it is classical that the "last invariant curve" is not C°°, 
even if the dynamic is C°° : if this happens, by using K.A.M. theorems the curve 
cannot disappear. . . 

Secondly, M. Herman built in [9j some C"^ symplectic twist maps that have a C^- 
invariant curve on which the dynamic is conjugated to the one of a Denjoy counter- 
example; such a curve cannot be C^. In [7], J. Mather asks if such a counter-example 
exists. 

To build our counter-example, we will use a family of symplectic twist maps that 
was introduced by M. Herman in [9j. These maps are defined by : 

f^ : T X R T X R; {e,r) ^ (9 + r,r + (f{9 + r)). 

^The precise definition of C-^ in this context will be given in subsection l2.1l 
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where : T — )• M is a C"^ map such that Lp{6)d9 = 0. 

As noticed by M. Herman, the main advantage of this map is the following one. We 
denote a lift of 5 : T — t- T by 5 : M — t- R. Then the graph of : T — )• R is invariant by 
ftp if and only if we have: 

• g = Idj + is an orientation preserving homeomorphism of T; 

• IdR + = i {g + g'^). 

Hence, in order to answer to Mather's question, we just have to find g = Idj+ip : T — )• T 
that is an increasing non-differentiable homeomorphism of T with no periodic points 
such that g + g^^ is C^. 

We begin by using a Denjoy counter-example (see [8j and [9] for precise construc- 
tions) g = Idf + ij) : T — )• T. The non- wandering set of g is then a Cantor subset 
K oi T such that g\^ is minimal. We then consider a point xq G T\K that is not 
in the Cantor subset K and its orbit {xk)k& = {g^{xQ))k£Z- Then we modify g in a 
neighborhood of this orbit in such a way that the new homeomorphism h : T — )• T 
coincides with g along the orbit of xq and is at every point but the orbit of xq. At 
every point Xk of the orbit of xq, we assume that h has some left and right derivatives, 
denoted by /3[ and such that: VA; G Z, + = Pi + 

If now we define : T — )• M by : Id^^ + ^93 = \ + h^^^, then 93 is at 
every point of T but the orbit of xq, has a right and left derivative along the or- 
bit of xq and verifies (we denote by (^^ and the right and left derivative of (p): 

S Z, ip'^(xk) = f^l ~^ ^ ~ /^fc + 2 = ip'^{xk). Hence ip is differentiable. 

Roughly speaking, the left and right derivatives of h at x^ and Xk-i are balanced in 
the formula that gives (p. This idea that the irregularities of h and are balanced 
in the formula that gives (p was the one that used M. Herman in [9] to construct his 
Denjoy counterexample for a symplectic twist map. 

If we choose carefully h, we will see that ip is in fact C^. Let us point out that 
things are not as simple as they seem to be, and the choice of the sequences (/3^) and 
(/3p is a delicate process as we will explain in the next section. 

2 Proof of theorem [1] 

2.1 Generalities about twist maps and other topics 

Notations. • T = R/Z is the circle. 

• A = T X R is the annulus and an element of A is denoted by (6, r). 

• A is endowed with its usual symplectic form, oj = dO f\dr and its usual Riemannian 
metric. 
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• vr : T X M T is the first projection and vr : R its lift. 

• if a € T, Ra : T — 7- T is the rotation defined by Ra{G) = + a. 

Definition. A diffeomorphism / : A — )• A of the annulus that is isotopic to identity 
is a positive twist map (resp. negative twist map) if, for any given hft / : — )• 
and for every 6 G M, the maps r i— )• vr o /(0,r) is an increasing (resp decreasing) 
diffeomorphisms. A twist map may be positive or negative. 

Then the maps that we defined at the end of the introduction are positive 
symplectic twist maps. 

Definition. Let 7 : T — )- M be a continuous map. We say that 7 is C-*^ at 6* € T is 
there exists a number 7'(0) G M such that, for every sequences (0^) and (0^) of points 
of T that converge to 6 such that 9j^ ^ 6'^, then: 

where we denote by 6\ — 9"^ the unique number that represents 9\ — 9^ and that belongs 
to]-i,i]. 

If we assume that 7 is differentiable at every point of T, then this notion of 
coincides with the usual one (the derivative is continuous at the considered point). 



2.2 Denjoy counter-example 

Following [9j p. 94, we define a Denjoy counter-example in the following way. 
We assume that a ^ Q/Z , 6 > and that C >> 1. Then we introduce: 
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ac 



i\k\ + c)(iog{\k\ + c)y+s 

where ac is chosen such that ^^^A: = 1- We use a function r] 



such that 



r/ > 0, support(7y) C [|, |] and Jq r]{t)dt = 1. We define % by : %(t) = t] yj^j- Then 
we have: Jq r]k{t)dt = l^. Moreover, there exist two constants Ci, C2, that depend 



only on rj, such that : 



Ci < \m\ < C2; ^< Wk\ < 



"4' 



We assume now that C >> 1 is great enough so that: 



Vfc G 



4 



1 



Co < 1. 
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Then the map gt : [0,4] ^ [0,4+i] defined by gk{x) = (l + (%±i - l) dt 
is a C°° diffeomorphism such that gk{^k) = ^fc+i- 

There exists a Cantor subset C T that has zero Lebesgue measure and that is 
such that the connected components of T\K, denoted by {Ik)k<=z, are on T in the same 
order as the sequence (ka) and such that length(/fc) = i^. 

Let us recall what is the semi-conjugation j : T — t- T of the Denjoy counter-example to 
the rotation Ra- II x G {ka; k G Z}, then we define : j~^{x) = Jq dfj,{t) where fj, is the 

probability measure ji = ^k^ka^ ^ka being the Dirac mass at ka. Then j : T — t- T 

k&Z 

is a continuous map with degree 1 that preserves the order on T and that is such that 
jih) = ka. 

Then there is a diffeomorphism (7 : T — )• T that fix K, is such that K is the unique 
minimal subset for g, has for rotation number p{g) = a, verifies jog = R^oj. If /c E Z, 
we introduce the notation: g\2^ = g^; then we have: gk{Ik) = -^fc+i- Following [9j again, 

we can assume that: g'f. = g'^j^ = + ~ ^) ° ^-^^ where R\^{Ik) = [0,4] 

and that gk : h h+i is defined hy : gt = R\k+^ o hk o R_x^ . 

Let us point out two facts that will be useful: lim Wg'j. — 1|| =0 and: 

\k\—^oo 

ye G K,g'(e) = i. 



2.3 Modification of the Denjoy counter-example g 

We choose xq G /q and we consider its orbit {xk)k& = {g^{xo))k<^i. 

Then we will build a perturbation h of g such that g^^ = h\K aiid \/k G Z, h{xk) = 

g{xk) = Xk+i- 

Notations. 

• for all A; G Z, we have: 4 =]afc,6fc[, =\ak,Xk] and Rk = [xk,hk[, 

• X : IK is defined by: x = 9 + 9^^ - 2/(iR; 

• for all A; G Z, we denote: ak = g'{xk) and nik = 2 + x'i^k)- 

Because of the definition of X: we have then: V/c G Z, H — = mk- 

— 1 

Notations. For every parameter m G M, let $m :]0, -|-oo[— )•] — 00, m[ be defined by : 
<^m{t) = m-\. 

Let us notice that every function is an increasing diffeomorphism. Moreover: 

• if m < 2, then has no fixed points and : Vt, ^mit) < t and lim ^^{t) = — 00; 

n— >+oo 

• if m = 2, 1 is the only fixed point of Moreover: if t > 1, then 1 < ^m{t) < t 
and lim $^(*) = 1; if i < 1, then ^m{t) < t and lim ^m{t) = -00; 
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if 771 > 2, $m has two fixed points, p_ < P+', if i > P+, then < <l*.m(i) < 
t and hm $^(0 =P+; if P- < t < p+, then p_ < t < ^m{t) < p+ and 

n— >+oo 

hm <J>m(*) = P+; if i < then ^m{t) < t and hm <^>J^(t) = -oo. 



n— >+oo 



We have: V/c e Z,afe = $mfe(afc-i)- 

Let us now choose /Jq > uq and /Sq^ > oq (each of them is then denoted by /3o). As 
every $m is increasing and defined on ]0, +oo[, we can define {Pn)n>o in the foUowing 
way: /3„+i = ^m„Wn)- Then Vn > 0,/3„ > a„ > 0. 

Lemma 1. We have: hm (3n = 1- 

n— f+oo 

Proof Let us recah that: hm a„ = 1. We deduce that hminf/3„ > 1 and that 

n— >+oo n— >'+oo 

hm m„ = 2. 

n— )-+oo 

Let us fix e > 0; then there exists > such that: Vn > N, nin < 2 + e. Then, for 
all n > iV, we have: /3„+i = #m„+i(/3n) = ^2+e(/3n) - (2 + e - m„+i) < $2+e(/3n)- 
Using the fact that $2+e is increasing, we easily deduce: Vn > 0,/3Ar+rt < 
We know that (<52-i-F(/3Af))nGN has a limit, and because liminf /3„ > 1 this limit cannot 

n— >+oo 

be smaller than 1. Hence lim <&2+e(/^™) = if we denote the greatest fixed point 



hCXD 



of <I>2+e by p+{e). We deduce that lim sup < We have: p+(e) = ^^^^V^^t ifl^ 

n— >+oo 

hence: lim p^(e) = 1. 

£-5-0+ 

Finally, we have proved that limsup/3„ < 1. As liminf /3„ > 1, we deduce the lemma. 

n— 5>+oo n— >-+oo 

D 

In a similar way, we choose < < a_i and < jS^^ < a^i and we denote each 
of them by /3_i. As every is increasing and defined on ] — oo,m[, we can define 
(/3fe)fc<_i by = (A). Then : V/c < -1,/?^ < a^. 

Lemma 2. We have: lim /3fc = 1. 

The proof of this lemma is similar to the one of lemma [TJ Now, we can assume: 

• /3o" + ^ = /3o^ + ^- 

Then we denote this last quantity by rho = l^n + = f^n + -^ir- Because /3o > ao 

p_i p_i 

and < q;_i, we necessarily have: rho > ""^o- 

Now (/3^) (resp. is a good candidate to be the left (resp. right) derivative of 

h along the orbit (x^). 
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Proposition 1. There exists an orientation preserving homeomorphism /i : T — )• T 
such that: 

• h^K = 9\K and Wk G Z, h{xk) = g{xk); 

• h and h^^ are at every point ofT but the orbit of xq; 

• h and h^^ have some right and left derivative at every point Xk of the orbit of xq, 
and h'^{xk) = h'^{xk) = Moreover, h^ji^^ and h^i^ are . 

Remark. If proposition [1] is true, tlien theorem [1] is proved: ii h = Idj + ^ and 
ip = g + g^^ — 2/(iK, then the graph of ip is invariant by and the dynamic of 
restricted to this graph is the one of a Denjoy counter-example with a as rotation 
number. Moreover, ^p is non-differentiable along the orbit of xq but is C^. Indeed, as 
g and are, (p is at every point of T but the orbit of xq. Moreover, the restriction 
of ip to each interval =]ak,Xk] or Rk = [xk,bk[ is C^. To prove that ip is C^, we 
then just have to prove that the right and left derivatives are equal along the orbit of 
xq. We have: 

. if A; / 0, ^'^ixk) = /3^ + ^- 2 = mfc-2 = /3|« + ^- 2 = ^'^{xk); 

• if = 0, v9'^(xo) = + ^ - 2 = mo - 2 = /3o^ + ^ - 2 = ip'^{xo). 
Hence (p is C^. 

Let us now prove proposition [TJ We modify g, or rather its derivative, in each 
interval L/. and i?^ in the following way. Let us notice that: lim l^fr , i — 1| =0; 

|fc|->+oo 

lim = lim /3f = 1; g' = 1. 

>' + 00 |fc|— f+oo ' 

Then on each interval Lk =]ak, Xk], we replace g' by a continuous function 6k :]afc, Xk] — t- 
MX such that: 

1. 6k{xk) = (3^; 

2. 6k coincide with g' in a neighborhood of o^; 

3- Q = Q g' = ~g{xk) - ff(afc); 

4. Vt e Lk, \6k{t) - 1| < max{|g|i^ - 1|, - 1|} + 

To build 5k, we just have to replace g' between Xk — £k and Xk by some affine function 
and then to modify slightly g' elsewhere in Lk to rectify the value of the integral. If 
Ek is small enough, than the change in the integral is very small and we have the last 
inequality (but of course the slope of the affine function can be very great, so the 
perturbation of g that we build in not small in topology). 
We then define h^i^ by: 

Vt e [ak,Xk],h{t) = g{ak) + / 6k{s)ds. 
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We proceed in similar way to define h^R^. and we obtain similar properties. Moreover, 
we ask: h^j^ = g\x- 

Then h is continuous. By construction, its restriction to every interval is contin- 
uous. Moreover, we have lim — g{ak)\ = (because of point[l]and the fact that 

fc|— >-+oo 

lim length(/fc) = 0). We deduce that h is continuous at every point of K. Moreover, 

|fc|— ^oo 

h is orientation preserving and injective by construction. Hence h is an orientation 

preserving homeomorphism of T. 

Moreover, h\x = 9\K by construction and VA; G Z, /i(xfc) = g{xk) by point [3l 

By construction, h\R^ and h\i^ are (and then the same is true for /i^"*^), h'^{xk) = 

(3^ and h'^ixk) = 

Let us now prove that h is differentiable at every point of K and that /ij^ = 1. 
We consider y £ K and a sequence (?/„) that converge to y and that is such that: 

Vn,y„ 7^ y. We want to prove that lim ^^"'^ = 1. Considering eventually 

n^+oo yn-y 

different cases, we can assume that (y„) tends to y from above. Then there are two 
cases: 

• either y = for some k. Then we have the conclusion by points 

• or y is accumulated from above by a sequence {ak„)n£N that are left ends of 
intervals Ij^,. 

Because g is and (7'^ = 1, for every e > there exists r] > such that for every 
z G [y,y + then \g'{z) — 1| < e. Let us assume that n is big enough such that 
y-n £ [y,y + 111- There are three cases: 

• yn £ K. Then there exists z £ [y,y + rj[ such that: 



KVn) - h{y) 



9{yn) - 9{y) 



\g'{z) - l| < e; 

yn-y 

• yn ^ Lk for some k. Then there exist z,z' G [y,y + ri[ such that: 

h{yn) - h{y) ^ Kyn) - Kak) + gjak) - g{y) ^ yj}_^z^h'{z) + ~ ^ g'jz') 
yn-y yn-y yn-y yn-y 

• yn & Rk for some k. Then there exist z,z' G [y, y + ??[ such that: 

h{yn)-h{y) Kyn) - h{xk) + g{xk) - g{y) yn-xk,,. Xk-y 
= = h {z) + g {z ). 

yn-y yn-y yn-y yn-y 

Because g is and because of point HI if rj is small enough, then h'{z) and g'{z') 
are close enough to 1, and their barycentre is close to 1 two. 
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Hence we have prove that h is derivable along K and that /i'^ = 1. 

Because g is and g'^j^ = 1, because of point [H h is on K and = 1. 
Then h satisfies all the conclusions of proposition [TJ 
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